A semiconductor memory device having a three-dimensional structure is proposed in which a memory hole is formed in a stacked body in which a plurality of electrode layers are stacked and a charge storage layer and a channel are provided in the memory hole. The charge storage layer has a function of trapping charges in the layer, the charges are transferred between the charge storage layer and the channel, and thus a write operation and an erasing operation are performed. However, improvements in miniaturization and operation characteristics of a memory cell in such a semiconductor memory device still remain desired.